2 research outputs found
Recommended from our members
Training Noise-Robust Spoken Phrase Detectors with Scarce and Private Data: An Application to Classroom Observation Videos
We explore how to automatically detect specific phrases in audio from noisy, multi-speaker videos using deep neural networks. Specifically, we focus on classroom observation videos that contain a few adult teachers and several small children (< 5 years old). At any point in these videos, multiple people may be talking, shouting, crying, or singing simultaneously. Our goal is to recognize polite speech phrases such as "Good job", "Thank you", "Please", and "You're welcome", as the occurrence of such speech is one of the behavioral markers used in classroom observation coding via the Classroom Assessment Scoring System (CLASS) protocol. Commercial speech recognition services such as Google Cloud Speech are impractical because of data privacy concerns. Therefore, we train and test our own custom models using a combination of publicly available classroom videos from YouTube, as well as a private dataset of real classroom observation videos collected by our colleagues at the University of Virginia. We also crowdsource an additional 1152 recordings of polite speech phrases to augment our training dataset. Our contributions are the following: (1) we design a crowdsourcing task for efficiently labeling speech events in classroom videos, (2) we develop a neural network-based architecture for speech recognition, robust to noise and overlapping speech, and (3) we explore methods to synthesize new and authentic audio data, both to increase the training set size and reduce the class imbalance. Finally, using our trained polite speech detector, (4) we investigate the relationship between polite speech and CLASS scores and enable teachers to visualize their use of polite language
Recommended from our members
Birds and Trees of Benjamín Aceval
This project developed ecotourism in Benjamin Aceval in order to benefit Hotel Cerrito at Escuela Agricola San Francisco. Potential ecotourism attractions were investigated through consultation with experts, observations in situ, and independent research. The group cataloged 45 tree species (accessible by QR code), created a self-guided tour of 10 signature trees, produced 1 interpretive sign for bird observation that doubles as a model, and delivered an application that profiles 50 birds. These deliverables are first steps to transform Benjamin Aceval into an ecotourism destination. Recommendations include: finish the tree inventory and bird tour, increase the number of tree tours, and expand the application